An augmented reality (AR) technology is known in which a virtual image is displayed to be overlaid at a certain position in an image in real space. Such an AR technology is becoming widely used in applications, one of which is, for example, to support working of a worker in workspace, such as a factory, by displaying work support information indicating the work content, a working object location, and the like to be overlaid on a captured image.
In the AR technology, a technique to estimate the position and orientation of a camera in real space is used in order to overlay a virtual image at a suitable position on a captured image. In addition, in a situation where the user is working, the position and orientation of a camera frequently changes. Therefore, the position and orientation estimation is prone to temporary failure. From this, a return processing technique by which, from a state where the position and orientation estimation has failed, the estimation is resumed is desirable.
As an example of return processing methods, a method has been proposed in which feature points in the current image and feature points in a key frame are correspondingly related to each other and, based on coordinate information of a three-dimensional map for the correspondingly related features in the key frame, the position and orientation of a camera at the time of capturing the current image is estimated.
In addition, a position and orientation measurement apparatus as described below that inhibits calculation of the position and the orientation from collapsing has been proposed. The position and orientation measurement apparatus generates a plurality of positions and orientations based on a plurality of positions and orientations obtained from the previous frames, determines the position and orientation of a camera by performing a non-linearity optimization of each of the plurality of positions and orientations, and selects the most suitable position and orientation from among the determined positions and orientations. As related art documents, Japanese Laid-open Patent Publication No. 2008-176509 and J. Straub et al., “Fast relocalization for visual odometry using binary features,” In Proc. ICIP, pp. 2548-2552, 2013 are disclosed.